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Abstract 

The purpose of this qualitative research study is to analyze instructors’ exam questions at a 
Primary Education Department in terms of the exam’s period, the comprehensibility of the 
instructions, cognitive level, and the appropriateness to the critical thinking. This qualitative 
study is based on document analysis method. 100 randomly selected exam papers and 1665 
questions asked in these exams are analyzed by three experts in the field. The results conclude 
that the exam questions are generally at knowledge level in terms of cognitive domain and 
they are not appropriate to critical thinking. 
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Introduction 

The use of critical thinking skills is vital for people to fully participate in democratic 
processes of their communities. Therefore, educating people to have critical thinking abilities 
with the intellectual tools and capacities is a target to be reached for democratic societies. It is 
possible to reach the objective by educating children as critical thinkers at early ages. 
Duckwort (1964, p. 172) states that education systems have two goals. The first one is to 
educate individuals who have the capacity of generating new ideas rather than repeating the 
previous works of researchers and scientists. The second one is to educate individuals with 
critical thinking capacities in place of individuals who accept everything without questioning. 

Teaching the ability of critical thinking to primary school children is robustly 
associated with the intellectual tools and critical thinking capacities of their primary school 
teachers. Consequently, it is important that education of primary school teachers should focus 
on critical thinking skills with all dimensions. One of these dimensions is the appropriateness 
of the questions given to prospective primary school teachers in the midterm and final exams. 

Critical Thinking 

§ahbat (2002, p.14 cited in ip§iroglu 1998) expresses that students accustomed to rote 
learning and conveying information are shocked when they are asked to comment on a written 
work, poem, etc., with their own words. Since they do not know how to think on a given topic 
and they do not grasp the importance and requirement of this process, they quickly transform 
this into a simple buying and selling process and find someone who thinks for themselves. 
This can be explained as the unfamiliarity of our culture to the critical thinking. 

Historically, critical thinking can be traced back as far as Socrates, and has developed 
through the centuries, via the writings and teachings of such renowned scholars as Aquinas, 
Aristotales, Marx, members of the Frankfurt School, etc. Scientists such as Robert Boyle and 
Sir Isaac Newton developed and used critical processes of thought that challenged the 
accepted views of the world and demanded a rigorous framework based on carefully gathered 
evidence and sound reasoning. The contribution of twentieth century educational philosophers 
such as Dewey, Wittgenstein and Piaget has been to highlight the importance of education in 
fostering critical thinking abilities, in order to challenge prejudice, over-generalization, 
misconceptions, self-deception, rigidity and narrowness (Hargreaves & Grenfell, 2003). 

The 1990 Delphi Report on critical thinking, endorsed by an expert panel from a 
variety of disciplines, defined critical thinking as “purposeful, self-regulatory judgment which 
results in interpretation, analysis, evaluation, and inference, as well as explanation of the 
evidential, conceptual, methodological, criteriological, on contextual considerations upon 
which that judgment in based...” (Facione, 1990, p. 3). According to Angelo (1995, p.6) most 
formal definitions characterize critical thinking as the intentional application of rational, 
higher order thinking skills, such as analysis, synthesis, problem recognition and problem 
solving, inference and evaluation. 

In the mid-1990s, a large sample of California faculty members affirmed the 
importance of critical thinking as an educational outcome. Eighty-nine percent of the faculty 
said critical thinking was a “primary objective” of their teaching-but only %19 could give a 
clear explanation of the concept. While %78 of these faculty said that students “lacked 
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standards” to assess their own thinking, only %8 could name or describe any of these 
standards (Paul, Elder, & Bartell, 1997). 

Critical thinking can be defined as a complex activity and it is not true to expect a 
single method of instruction will prove sufficient utility for developing each of its components 
parts. Whatever methodology is used, it is unquestionable the effect of the questions 
organized in fostering the critical thinking abilities. 

Questions and Their Classification 

Questioning is accepted as a methodology which triggers thinking. Thinking occurs 
when people have question marks in their minds. People need questions in order to use one’s 
life (Ozden, 1999, p. 106). Giirses et al. (2005, p.363) affirm that questioning is the most 
essential step for the activity of thinking. Questioning can be admitted as a tactic that activates 
thinking. In any environment in which the action of thinking happens, learning occurs in its 
real meaning. As Hussain mentions (2003 cited in Ellis, 1993 and Foster, 1983) in the realm 
of teaching and learning, questions have been cited as not only the most often used, but also 
the single most important strategy used by instructors. The researches ground on classroom 
implementation throughout the 20 th century consistently document that the presentation and 
memorization methods are foremost in these classrooms (Onosko, 1988, p.l). Research over 
the last sixty years has shown that, of teachers' questions, the predominating ones are those 
that are concerned with simple data and recall of facts already learned which fall under lower 
order questions (Hussain, 2003). 

Questions can and have been used for a wide variety of educational purposes: 
reviewing previously read or studied material; diagnosing student abilities, preferences, and 
attitudes; stimulating critical thinking; managing student behavior; probing student thought 
process; stirring creative thinking; personalizing the curriculum; motivating students; and 
assessing student knowledge (Sadker, 2003). Teachers use questions for these reasons at 
different stages of education. When the questions given in the exams are considered, 
“assessing student knowledge”, “probing student thought process”, “stimulating critical 
thinking” can be listed as primary goals. 

Akbulut (1999, p. 16-17) asserts that teachers can learn which questions can be asked 
at the beginning of the lesson, at the practice stage of the lesson and at the wrap up stage of 
the lesson by knowing the classification of the questions rather than gaining this knowledge 
with the experience throughout years. Besides, the questions should also be sorted in order to 
address the students into appropriate thoughts. The categorization of the questions causes the 
teachers to gain experience in developing new teaching materials and exposing the students’ 
previous knowledge (Hadder, 1970, p.93). Although the initial efforts on the categorization 
of the questions are mostly accepted in the field, some researchers criticize these 
categorizations in some ways and try to form new categorizations in the following years. 
Cognitive domain is the primarily discussed realm and new categorizations are added to this 
area. Yirksel (2007, p. 480) indicates that the primary extensive efforts were commenced in 
1948 on the gradual categorization of the objectives. A group of researchers working at the 
higher education institutions in the USA gathers in Boston with the purpose of forming a 
categorization that can be accepted by everyone. Even though the basic aim is to form a 
categorization of all fields, only the cognitive domain is classified within this period (Bloom, 
1956). The process that starts as the categorization of the objectives changes into the 
categorization of the questions. 
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Yiiksel (2007) states that alternative categorizations based on Bloom’s Taxonomy 
(1956) aims to formulate the Bloom’s Taxonomy as truer and accurate. Some of the 
alternative categorizations propound against Bloom’s Taxonomy are listed as follows: 
Categorization of Gerlach and Sullivan, Categorization of De Block, Categorization of 
Tuckman, Categorization of Williams, Categorization of Hannah and Michaelis, 
Categorization of Gagne and Briggs, Categorization of Stahl and Murphy, Categorization of 
Romizowski, Categorization of Quellmalz and Categorization of Haladayna. 

Bloom's Taxonomy 

Bloom Taxonomy is the most common approach employed in categorization of the 
question levels and educational objectives. Bloom Taxonomy consists of six levels that are 
hierarchically aligned from low cognitive skills to high cognitive skills. These levels are 
Knowledge, Comprehension, Application, Analysis, Synthesis, and Evaluation. 

Bloom's Knowledge level requires an answer that demonstrates simple recall of facts. 
Questions at this level could ask students to answer who and what and to describe, state, and 
list. Comprehension requires an answer that demonstrates an understanding of the 
information. Questions at this level might ask students to summarize, explain, paraphrase, 
compare, and contrast. Application requires an answer that demonstrates an ability to use 
information, concepts and theories in new situations. Questions at this level may ask students 
to apply, construct, solve, discover, and show. Analysis requires an answer that demonstrates 
an ability to see patterns and classify information, concepts, and theories into component 
parts. Questions at this level could ask students to examine, classify, categorize, differentiate, 
and analyze. Synthesis requires an answer that demonstrates an ability to relate knowledge 
from several areas to create new or original work. Questions at this level might ask students to 
combine, construct, create, role-play, and suppose. Finally, Evaluation requires an answer that 
demonstrates ability to judge evidence based on reasoned argument. Questions at this level 
may ask students to assess, criticize, recommend, predict, and evaluate (Duron, Limbach & 
Waugh, 2006, p. 160). Although the Bloom Taxonomy is primarily developed for the 
classification of educational objectives, it has been widely used in many researches for the 
analysis of the questions asked by the teachers in oral and written exams and the questions in 
course books as well. 


Bloom's Taxonomy and Critical Thinking 

Critical thinking has been an important issue in education for many years. After the 
1948 Convention of the American Psychological Association, Benjamin Bloom took the lead 
in developing “the goals of the educational process,” including knowledge, comprehension, 
application, analysis, synthesis, and evaluation. Critical thinking in education has been hotly 
debated since then (Schneider, 2002). Bloom’s taxonomy does not explicitly define critical 
thinking. Rather, it includes six knowledge levels that constitute the construct of critical 
thinking (Aviles, 2000, p.4). Bloom (1956, p.46-47) acknowledged critical thinking as a broad 
aim of education and stated that such broad aims are helpful in suggesting general policy and 
direction for curriculum development. 

The theory of critical thinking began primarily with the works of Bloom (1956), who 
identified six levels within the cognitive domain, each of which related to a different level of 
cognitive ability. Knowledge focused on remembering and reciting information. 
Comprehension focused on relating and organizing previously learned information. 
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Application focused on applying information according to a rule or principle in a specific 
situation. Analysis was defined as critical thinking focused on parts and their functionality in 
the whole. Synthesis was defined as critical thinking focused on putting parts together to form 
a new and original whole. Evaluation was defined as critical thinking focused upon valuing 
and making judgments based upon information (Duron, Limbach & Waugh, 2006, p. 160). 
MacPherson and Mansfield (2008) affirm that critical thinking is inherent in Bloom’s 
Taxonomy. In addition, they assert that top three levels of Bloom’s Taxonomy (analysis, 
synthesis, and evaluation) are associated with critical thinking. Blank-Libra (1997, p.17 cited 
in Gall 1984) provides evidence to support the notion that higher-level questions will provoke 
higher-level responses from students. The same principle, of course, applies to lower-level 
questions. Bloom (1988) says that his graduate students have done a series of studies that 
have supported the same idea. 

The research studies conducted on the analysis of the questions in Turkey largely 
focused on exam question used in various courses and questions figured in the course books at 
primary schools and secondary schools or questions asked in national exams (Ccpni & Azar, 
1998; £epni, Kele§ & Ayvaci, 1999; Gelen, 1999; £epni, Ayvaci & Kele§, 2001; Koray & 
Yaman, 2002; Tekin & Ayas, 2002; Akpinar, 2003; Ccpni, 2003; Ccpni, Ozsevgcnc & 
Gokdere, 2003; Karamustafaoglu et al., 2003; Mutlu, U§ak & Aydogdu, 2003; Sagir, 2003; 
Giiler, Ozek & Yaprak, 2004; Azar, 2005; E§, 2005; Giirses et al., 2005; Karaman, 2005; 
Kogce, 2005; Ya§ar, 2005; Akpinar & Ergin, 2006; Baysen, 2006; Dindar & Demir, 2006; 
Ozmen & Karamustafaoglu, 2006; Ozgiir, 2007; Erman, 2008; Kogce & Baki, 2009). It is 
striking that document analysis is employed as a research methodology in most of these 
studies, and observations, interviews and questionnaires are applied in some other studies as 
well. It is necessary to indicate that there are scarcely any research analyzing exam questions 
at the University level and even no research is found about the questions given at the primary 
teacher training departments within the related literature. Related studies conclude that most 
of the analyzed questions only concentrate on levels such as knowledge, comprehension, and 
implementation that do not require high levels of thinking and few almost no questions are 
asked appropriate to the critical thinking skills demanding analysis, synthesis, and evaluation 
levels. 


Method 

In this qualitative study, document analysis method was employed. In this method, any 
part of a selected text or document was analyzed and the features of the text transformed into 
numerical data in order to utilize any statistical operation. 

Research Group 

The exams given to the prospective primary school teachers at an Education Faculty 
was the main data source. 100 randomly selected exam papers and 1665 questions asked in 
those exams were used as the sample of this study. 

Data Collection Tools 

The features of the Bloom Taxonomy’s cognitive levels were primarily designated by 
reviewing the related literature to determine the cognitive levels of the questions given to the 
prospective primary school teachers and their appropriateness to the critical thinking. Thus the 
criteria for the evaluation of the questions were formed. 
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1665 question obtained from 100 randomly selected exam papers were analyzed by a 
research team involving the researchers and two experts in the field. The criteria formed for 
the purpose of the study and the cognitive level of the questions including the knowledge, 
comprehension, application, analysis, synthesis, evaluation levels and appropriateness to 
critical thinking were used. The questions’ appropriateness to critical thinking was examined 
in regard to the first and last three levels of the Bloom Taxonomy. The obtained data was 
tabulated into frequency and percentage distribution by using the SPSS program. 

Findings 

The study year of the course which the exam questions were analyzed, are showed in 
Table 1. 

Table 1 

The Study Year of the Course Which the Exam Questions were Analyzed. 


The year of the course f 

% 

First Year Course 

35 

35,0 

Second Year Course 

34 

34,0 ' 

Third Year Course 

16 

16,0 

Fourth Year Course 

15 

15,0 

Total 

100 

100,0 


As it is demonstrated in table 1, 69 % of the analyzed exams questions belonged to the 
courses of the first and second year. The exam questions of the third and fourth year courses 
constitute 31 % of the total sample. The occurrence time of the exams, are showed in Table 2. 

Table 2 

The Occurrence Time of the Exams 


The occurrence time of the exams / 

% 

Mid-term exam 

49 

49,0 

Final exam 

51 

51,0 

Total 

100 

100,0 


When the table 2 is examined, it can be observed that 51 % of the analyzed exams are 
final exams and 49 % of them are Mid-Term exams. Length of the exams, are showed in 
Table 3. 
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Table 3 

Length of the Exams 


Length of the exams / 

% 

0-20 min. 

10 

10,0 

21-40 min. 

50 

50,0 

41-60 min. 

39 

39,0 

61 min and over 

1 

1,0 

Total 

100 

100,0 


Table 3 demonstrates that 50 % of the analyzed exams take place in 21-40 minutes, 
39% of them are limited with 41-60 minutes. It is remarkable that 0-20 minute the exams have 
only 10 % distribution in the overall sample and the longest exams in terms of their 
implementation time have just 1 % in the distribution. It is clear that 89 % of the exams of the 
prospective primary school teachers are held in 21-60 minutes. The comprehensibility level of 
the exams’ instructions, are showed in Table 4. 

Table 4 

The Comprehensibility Level of the Exams’ Instructions 


Clearness of the instructions!' % 

Totally Clear 

76 76,0 

Poor 

24 24,0 

Total 

100 100,0 


Table 4 represents that the 76 % of the instructions are totally clear and 
comprehensible whereas 24 % of them are poor and difficult to understand. The 
comprehensibility of the exam instructions that is one of the basic requirements for the 
implementation easiness is fundamental in improving the students’ success in the given exam. 
Therefore, it is a noteworthy result that approximately one fourth of the exam instructions are 
vague and problematic in terms of their comprehensibility. The cognitive level of the 
analyzed exam questions, are showed in Table 5. 
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Table 5 

The Cognitive Level of the Analyzed Exam Questions 


Cognitive Level 

f % 

Knowledge 

965 58,0 

Comprehension 

237 14,2 

Application 

248 14,9 

Analysis 

137 8,2 

Synthesis 

48 2,9 

Evaluation 

30 1,8 

Total 

1665100,0 


The analysis in table 5 presents that the exam questions are mostly at the knowledge 
level with a 58 % regarding the Bloom’s Taxonomy cognitive levels. It is an appealing 
outcome that more than half of the questions given to the prospective primary school teachers 
are at knowledge level. The questions at comprehension and application level have both 14 % 
in overall distribution. The questions at the analysis level have higher proportion than the 
questions at synthesis and evaluation level with an 8.2 percentage. The total distribution of the 
question at synthesis level and evaluation level is less than 5 % of the whole distribution. The 
distribution of the questions at evaluation level, which is considered as the top stage among 
these hierarchical levels with a 1.8 percentage, awakes the question of how the prospective 
primary school teachers are educated. Furthermore, the results of the present study match up 
with the outcomes of the previous studies in the literature. 

It is necessary to ask high cognitive level questions to enable prospective student 
teachers to think in a multifaceted way. Therefore, they can avoid the tendency of superficial 
thinking that they get used to by answering cognitive level questions. It is obvious that 
assessment of students’ success is one of the most important tasks of the teachers or 
instructors. The exams including questions with a high level thinking skills can be used as 
well as an assessment tool and a teaching material. The appropriateness level of the analyzed 
questions to critical thinking, are showed in Table 6. 

Table 6 

The Appropriateness Level of the Analyzed Questions to Critical Thinking 


Appropriateness to Critical Thinking 

f 

% 

Inappropriate to Critical Thinking 

1450 87,1 

(Knowledge - Comprehension - Application) 


Appropriate to Critical Thinking 

215 

12,9 

(Analysis - Synthesis - Evaluation) 



Total 

1665 

100,0 
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Table 6 illustrates that the proposition of the questions supporting the Knowledge 
level, Comprehension level and Application level is 87, 1 % in the overall distribution. The 
distribution of questions requiring high level thinking at analysis level, synthesis level and 
evaluation level form 12,9 % of the all questions. 

Of course, only considering the questions given at the exams as the primary source of 
education of prospective teachers as critical thinkers would be a mistake. However, exams in 
which the students perform their intake as an outcome offer unique opportunities for 
understanding of the development of critical thinking. Curses et al. (2005, p.366) asserts that 
preparation and evaluation of the questions involving analysis, synthesis and evaluation levels 
which are effective in improving the high level thinking skills is more difficult than preparing 
questions partaking at the Knowledge, Comprehension and application level. This can be an 
explanation of not preparing these kinds of questions or the critical thinking skills can be 
underestimated in the given courses. 

Conclusions and Recommendations 

There is no doubt that critical thinking skill is a treasure that every individual should 
have in order to overcome the dilemmas of globalization and information society. Therefore, 
the educating students with the resourses and strayegies of critical thinking starting from 
primary school level is essential for the development of true participatory democracy. Primary 
school teachers who are good at using critical thinking skills and have the knowledge of 
methodologies in conveying these skills to their students play an important role in this 
democratization process. 

The questions given in the exams by the instructors reflect the objectives, goals, 
outputs and the methodologies that the instructors apply in their teaching. The results of this 
study have strong similarities with the other studies conducted in Turkey about the 
appropriateness of questions in regard to the levels of critical thinking. Both the teachers in 
primary and secondary schools and the university instructors tend to check whether the 
students memorize the decontextualized information by using semester exams and they do not 
force the student enough to critically analyze, synthesize and evaluate what they have learnt 
because of the low cognitive level questions in the exams. It should not be forgotten that 
prospective teachers have the tendency of using the same teaching methodologies and same 
kind of questions that they encounter during their university education, when they become 
classroom teachers. 

It is not surprising to discover that the evaluation of the students’ learning with low 
cognitive level questions in primary and secondary schools as well as in higher education 
institutions is a common assessment strategy. Once prospective teachers graduate from their 
programs without attaining high-level cognitive questions during their education, they do not 
prefer to assess their students’ progress with high-level cognitive questions as a teacher. 
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